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Abstract 

In this article, we consider a fundamental decentralized optimal control problem, 
which we call the two-player problem. Two subsystems are interconnected in a nested 
information pattern, and output feedback controllers must be designed for each sub- 
system. Several special cases of this architecture have previously been solved, such as 
the state-feedback case or the case where the dynamics of both systems are decoupled. 
In this paper, we present a detailed solution to the general case. The structure of 
the optimal decentralized controller is reminiscent of that of the optimal centralized 
controller; each player must estimate the state of the system given their available in- 
formation and apply static control policies to these estimates to compute the optimal 
controller. The previously solved cases benefit from a separation between estimation 
and control which allows one to compute the control and estimation gains separately. 
This feature is not present in general, and some of the gains must be solved for simul- 
taneously. We show that computing the required coupled estimation and control gains 
amounts to solving a small system of linear equations. 

I Introduction 

Many large-scale systems such as the internet, power grids, or teams of autonomous ve- 
hicles, can be viewed as a network of interconnected subsystems. A common feature of 
these applications is that subsystems must make control decisions with limited informa- 
tion. The hope is that despite the decentralized nature of the system, global performance 
criteria can be optimized. 

In this paper, we consider a particular class of two-player decentralized control prob- 
lems, illustrated in Figure 1, and develop the optimal controller in this framework. In 
recent years, much work has gone into developing results which characterize exactly when 
the standard linear-quadratic optimal control problem has a linear optimal controller. 
The problem we consider fulfils these criteria, but beyond the fact that the optimal con- 
troller is linear, very little has been hitherto known about its structure. This particular 
configuration is therefore chosen with some care, to be the simplest possible directed 
graph with measurement feedback that admits a linear optimal controller. 




Figure 1: Decentralized interconnection 
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Figure 1 shows two plants Gi and Q 2 , with associated controllers C\ and C 2 . Con- 
troller C\ receives information only from plant £/ 1; whereas C2 receives information from 
both Q\ and Q 2 - The actions of C\ affect both Q\ and G2, whereas the actions of C 2 only 
affect Q 2 . Furthermore, Gi affects Q 2 , but not vice- versa. In other words, information only 
flows from left-to-right. This is the simplest nontrivial structure where linear controllers 
are optimal for linear plants. This feedback interconnection may be represented using the 
linear fractional transformation, as shown in Figure 2. 



z - 




Figure 2: General four-block plant with controller in feedback 

In doing so, the graph structure results in a sparsity structure for both V and K. 
Specifically, we find the following block-lowcr-triangular sparsity pattern 
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while Vn, V12, and V21 are full in general. We assume the plant and controller are finite- 
dimensional continuous-time linear time-invariant systems. The goal is to find an %2- 
optimal controller K subject to this structural constraint. We paraphrase our main result, 
found in Theorem II. Consider the state-space dynamics 
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where w and v are white noise. The objective is to minimize the quadratic cost of the 
standard LQG problem, subject to the constraint that 

Player 1 measures j/i and chooses U\ 
Player 2 measures yi , 7/2 and chooses 112 

We show that the optimal controller is 



Kx 



|j/li«C 




H J 



where x\ yi U( denotes the optimal estimate of a; using the information available to Player 1, 
and x\ y<u is the optimal estimate of x using the information available Player 2. The ma- 
trices K, J, and H are determined from the solutions to Riccati and Sylvester equations. 
The precise meaning of optimal estimate and will be explained in Section V. Our 
results therefore provide a type of separation principle for such problems. Questions of 
separation are central to decentralized control, and very little is known in the general 
case. Our results therefore also shed some light on this important issue. Even though the 
controller is expressed in terms of optimal estimators, these do not all evolve according 
to a standard Kalman filter, since Player 1, for example, docs not know u 2 - 
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The main contribution of this paper is an explicit state-space formula for the optimal 
controller, which was not previously known. The realization we find is generally minimal, 
and computing it is of comparable computational complexity to computing the optimal 
centralized controller. The solution also gives an intuitive interpretation for the states 
of the optimal controller. The analysis of stabilization of such structured systems is also 
a significant contribution of this paper. We give a state-space parameterization of all 
stabilizing decentralized controllers for this problem. 

The paper is organized as follows. In the remainder of the introduction, we give a brief 
history of decentralized control and the two-player problem in particular. Then, we cover 
some background mathematics and give a formal statement of the two-player optimal 
control problem. In Section III, we characterize all structured stabilizing controllers, and 
show that the two-player control problem can be expressed as an equivalent structured 
model-matching problem, which is convex. In Section IV we state and prove our main 
result which contains the optimal controller formulae. We follow up with a discussion of 
the structure and interpretation of this controller in Section V. The subsequent Section VI 
gives an explicit construction of the controller. Finally, we conclude in Section VII. 

I-A Prior work 

If we consider the problem of Section I but remove the structural constraint on the 
controller, the problem becomes the well-studied classical H2 synthesis, solved for example 
in [30]. The optimal controller in this case is linear, and has as many states as the original 
system. 

The presence of structural constraints greatly complicates the problem, and the re- 
sulting decentralized problem has been outstanding since the seminal paper by Wit- 
senhausen [26] in 1968. Witsenhausen posed a related problem for which a nonlinear 
controller strictly outperforms all linear policies. Not all structural constraints lead to 
nonlinear optimal controllers, however. For a broad class of decentralized control prob- 
lems there exists a linear optimal policy, and finding it amounts to solving a convex 
optimization problem [5, 15, 17,24]. The two-player problem is in this class. 

Despite the benefit of convexity, the search space is infinite-dimensional since we must 
optimize over transfer functions. Several numerical and analytical approaches for address- 
ing decentralized optimal control exist, including [15,16,20,29]. One particularly relevant 
numerical approach is to use vectorization, which converts the decentralized problem into 
an equivalent centralized problem [18]. This conversion process results in a dramatic 
growth in state dimension, and so the method is extremely computationally intensive 
and only feasible for small problems. However, it does provide important insight into 
the problem. Namely, it proves that there exists an optimal rational controller for the 
two-player problem considered herein, and gives an upper bound on the state dimension. 
These results are presented in Lemma 13. 

A drawback of such numerical approaches is that they do not provide an intuitive ex- 
planation for what the controller is doing; there is no physical interpretation for the states 
of the controller. In the centralized case, we have such an interpretation. Specifically, 
the controller consists of a Kalman filter whose states are estimates of the states of the 
plant, and a static control gain that corresponds to the solution of an LQR problem. 
Recent structural results [1, 14] show that for general classes of delayed-sharing infor- 
mation patterns, the optimal control policy depends on a summary of the information 
available and a dynamic programming approach may be used to compute it. For linear 
systems in which each player eventually has access to all information, explicit formulae 
were computed in [8] . There are general results also in the case where not all information 
is eventually available to all players [27]. However, these results do not appear to easily 
translate to state-space formulae for linear systems. 
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Certain special cases of the two-player problem have been solved explicitly and clean 
physical interpretations have been found for the states of the optimal controller. Most 
notably, the state-feedback case admits an explicit state-space solution using a spectral 
factorization approach [22]. This approach was also used to address a case with partial 
output-feedback, in which there is output-feedback for one player and state-feedback 
for the other [23]. The work of [21] also provided a solution to the state- feedback case 
using the Mobius transform associated with the underlying poset. Certain special cases 
were also solved in [7], which gave a method for splitting decentralized optimal control 
problems into multiple centralized problems. This splitting approach addresses problems 
other than state-feedback, including partial output-feedback, and dynamically decoupled 
problems. 

In this article, we address the general two-player output-feedback problem. Our ap- 
proach is perhaps closest technically to the work of [22] using spectral factorization, but 
uses the factorization to split the problem in a different way, allowing a solution of the 
general output-feedback problem. We also provide a meaningful interpretation of the 
states of the optimal controller. This paper is a substantially more general version of 
the invited paper [11] and the conference paper [12], where the special case of stable sys- 
tems was considered. We also mention the related work [9] which addresses star-shaped 
systems in the stable case. 



II Preliminaries 



We use Z + to denote the nonnegative integers. The imaginary unit is j, and we denote 
the imaginary axis by jR. A square matrix A G R nx ™ is Hurwitz if all of its eigenvalues 
have a strictly negative real part. The set £ 2 (jM.), or simply £ 2 , is a Hilbert space of 
Lebesgue measurable matrix- valued functions T : jM. i-> C mxn with the inner product 
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trace(j"* (ju)G(joj)) dw 



such that the inner product induced norm ||J"j| 2 = (J 7 , J 7 ) 1 / 2 is bounded. We will some- 
times write £™ x ™ to be explicit about the matrix dimensions. As is standard, % 2 is a 
closed subspace of £ 2 with matrix functions analytic in the open right-half plane. H 2 is 
the orthogonal complement of H 2 in C 2 . We write TZ P to denote the set of proper real 
rational transfer functions. We also use 1Z as a prefix to modify other sets to indicate 
the restriction to real rational functions. So 1Z£ 2 is the set of strictly proper rational 
transfer functions with no poles on the imaginary axis, and TZ1-L 2 is the stable subspace 
of 1Z£ 2 . The set of stable proper transfer functions is denoted TZHoo . Every Q G 1Z p has 
a state-space realization 
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where Q* is the conjugate transpose of Q. If this realization is chosen to be stabilizable 
and detectable, then Q G TZHoo if and only if A is Hurwitz, and Q G 1Z'H 2 if and only 
if A is Hurwitz and D = 0. If z — Qw where Q G 'RM. 2 and w is white Gaussian noise 
with unit variance, we have the following well-known relationship between the norm of 
the signal z and the £2-norm of the system Q. 
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For the remainder of this paper, whenever we write \\Q\\ 2 , it will always be the case 
that Q £ 1ZW.2- For a thorough introduction to these topics, sec [30]. 

The plant V £ 1Z P maps exogenous inputs w and actuator inputs u to regulated out- 
puts z and measurements y. Our objective is to find a control law u = JCy where JC £ 1Z P 
so that the closed-loop map has some desirable properties. The closed-loop map is de- 
picted in Figure 2, and this figure corresponds to 
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The closed-loop map from w to z is given by the lower linear fractional transform (LFT) 
defined by F e (V,JC) = Vn+V 12 JC (I - V 22 ICy 1 V 21 . If JC = F t (J, Q) and J has a proper 
inverse, the LFT may be inverted according to Q — F U {J~ X ,JC), where F u denotes the 
upper LFT, defined as F U (M,JC) = M 22 + M 2 iJC (I - MuJCy 1 M 12 - 

We will express the results of this paper in terms of the solutions to algebraic Riccati 
equations (ARE) and recall here the basic facts. If D T D > 0, then the following are 
equivalent. 



(i) There exists X eR nxn such that 

A T X + XA + C T C - (XB + C J D)(D T D)- 1 {B T X + D T C) = 
and A - B(D T D)- 1 (B T X + D T C) is Hurwitz 

(ii) (A, B) is stabilizable and 



(2) 
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has full column rank for all uj £ 



Under these conditions, there is a unique X £ M nx " satisfying (i). This X is sym- 
metric and positive semidefinite, and is called the stabilizing solution of the algebraic 
Riccati equation. As a short form we will write (X, K) = ARE(A, B, C, D) where 
K = -(D T D)- 1 (B T X + D T C) is the associated gain. 



II-A Block-triangular matrices 

Due to the triangular structure of the problem considered herein, we also require nota- 
tion to denote the sets of block-lower-triangular matrices. To this end, suppose S is a 
commutative ring, m,n £ and m^n, > 0. We define lower ( S 1 , m, n) to be the set 
of block-lower-triangular matrices with elements in S partitioned according to the index 
sets m and n. That is, X £ lower (5, m, n) if and only if 

where X i:j £ S miXn > 

We sometimes omit the indices and simply write lower (S). We also define the matri- 
ces Ej = [I 0] T and Ej = [0 /] T , with dimensions inferred from context. For exam- 
ple, if we write XE\ where X £ lower (R, to, n), then E\ £ K("i+™2)xni _ -\Yh eil writing 
A £ lower (5), we allow for the possibility that some of the blocks may be empty. For ex- 
ample, if mi = then we encounter the trivial case where lower (S, to, n) — 5 ,m 2 x (m+™2) 
There is a correspondence between proper transfer functions Q £ 1Z P and state-space 
realizations (A, B, C, D). The next result shows that a specialized form of this correspon- 
dence exists when Q £ lower (TZ P ). 
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Theorem 1. Suppose Q £ lower (1Z P , fc, m), and a realization for Q is given by 
(A, B, C, D). Then there exists n eZ^ and an invertible matrix T such that 

T~ X AT £ lowcr(M, n, n) T~ 1 B £ lowcr(M, n, m) 

CT £ lowcr(M, k, n) D £ lowcr(K, k, m) 

Proof. Partition the state-space matrices according to the partition imposed by k and m. 
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Note that we immediately have D £ lower (R, k, m). However, A, B, and C need not have 
the desired structure. If G12 is empty, the sparsity pattern is trivial and any realization 
(A, B, C, D) will do. Suppose G12 is non-empty and let T be the matrix that transforms 
it into Kalman canonical form. 

There are typically four blocks in such a decomposition, but since G12 = 0, there can be 
no modes that are both controllable and observable. Apply the same T-transformation 
to G, and obtain the realization 
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This realization has the desired sparsity pattern, and we notice that there may be many 
admissible index sets n. For example, the modes A 55 can be included into either the An 
block or the A 22 block. These modes correspond to the modes of G21 that arc not in 
common with the modes of G11 or £22 ■ Also note that (3) is an admissible realization 
even if some of the diagonal blocks of A in this realization are empty. ■ 



II-B Stabilization 

We seek controllers JC £ 1Z P such that when they are connected in feedback to a 
plant V £ IZp as in Figure 2, the plant is stabilized. Consider the interconnection in 
Figure 3. We say that K, stabilizes V if the transfer function (w,Ui,U2) 1— > (2,2/1,2/2) is 
well-posed and stable. Well-poscdness amounts to I — 7 , 22(oo)/C(oo) being nonsingular. 
Similarly, if we consider the interconnection in Figure 4, we say that JC stabilizes 7^22 
if the transfer function («i,ti2) ^ (2/1 5 2/2) is well-posed and stable. The well-posedness 
condition is the same in this case as in the case of stabilizing V. The two notions of sta- 
bilization are discussed in [2-4,30] and are related via the following result. Stabilization 
can also be characterized using state-space, which we give in Proposition 3. 
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Figure 3: Feedback loop with additional inputs and outputs for analysis of stabilization 
of two-input two-output systems 
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Figure 4: Feedback loop with additional inputs and outputs for analysis of stabilization 

Proposition 2 (see [4, Lemma A.4.3]). Suppose there exists a /Co € 1Z P such that /Co 
stabilizes V . Then for any JC € 1Z P , JC stabilizes V if and only if JC stabilizes Vii- 



II-C Problem statement 

The problem addressed in this article may be formally stated as follows. Suppose V S TZ P 
is given. Further suppose that V22 € lower(7£ p , k, m). The two-player problem is 



minimize \\Vu + VnKil - V 2 2^)~ 1 'P2i || 2 
subject to JC € lower (1Z P , m,k) 
JC stabilizes V 



(4) 



We will also make some additional assumptions. We will assume without loss of gen- 
erality that V\\ and V22 are strictly proper and we will make some standard technical 
assumptions on V\ 2 and V 2 \ in order to guarantee the existence and uniqueness of the 
optimal controller. The first step in our solution to the two-player problem (4) is to deal 
with the stabilization constraint. This is the topic of the next section. 



Ill Stabilization of triangular systems 

In this section, we provide a state-space characterization of stabilization when both the 
plant and controller have a block-lower-triangular structure. Specifically, we find neces- 
sary and sufficient conditions under which a block-lower-triangular stabilizing controller 
exists, and we provide a parameterization of all such controllers akin to the Youla param- 
eterization [28]. 

The fundamental results regarding decentralized stabilizability of linear systems were 
developed in [25] via analysis of the system transfer functions. We present a state-space 
development of the triangular case, aimed at parameterizing all stabilizing controllers. 
We consider both the state-space and input-output characterizations of stabilization. 
Our approach closely parallels that in [30] for centralized systems, and we will use their 
notation whenever possible. Some of these cases have been addressed in [15] using a 
coprime factorization approach. Related work has also appeared in [19]. Many of the 
well-known equivalences between these two forms of stability are much more subtle in the 
decentralized case, and in this section we therefore carefully delineate these issues. 

Throughout this section, we assume the plant V has a minimal realization that is 
partitioned according to the partition imposed by the Vij ■ 
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(5) 



When dealing with the two-player problem, we further assume that the V22 subsystem 
is block-lower-triangular. Since we have V22 € lower(7\L p ), Theorem 1 implies that there 
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exists a minimal realization of V for which 
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(6) 



and we therefore assume that A, B 2l C 2 , -D22 have this form. Note that the block sizes 
must be consistent, so there exists n,m,k e I? + such that Ay e R"**™^ 5^. g jjruxm^^ 
Cij € R feiX ™3, and e R fe « xm j. A standard and useful characterization of stabilization 
is given in terms of state-space matrices, as follows. 



Proposition 3. Suppose V22 G Tip and K. <G 1Z P have the realizations (A, -B 2 ,C2, D22) 
and (Ak,Bk,Ck,Dk) respectively. Then the following statements are equivalent. 

(i) [C2,A,B2) and (Ck, Ak, Bk) are stabilizable and detectable, and K stabilizes V 22- 
(ii) I — D22DK is invertible and A is Hurwitz, where 
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Proof. That (i) implies (ii) follows from the standard result [30, Lemma 5.2], which states 
that if (C 2 ,A, B2) and (Ck,Ak,Bk) are stabilizable and detectable, then JC stabilizes 
V22 if and only if A is Hurwitz. To see the converse, notice that if (ii) holds, then 

^ is stabilizable 

It follows from the PBH test that (A, B2) and {Ak, Bk) are stabilizable. A similar argu- 
ment proves that (C 2 ,A) and (Ck,Ak) are detectable. Now applying [30, Lemma 5.2] 
again gives (i) as desired. ■ 

Proposition 3 is a characterization of stabilization for the 'P22 block. As stated in 
Proposition 2, we also require the existence of a stabilizing controller for V. The following 
result is standard [30, Lemma 12.1]. 

Proposition 4. Suppose V € 1Z P has a minimal realization given by (5). Then there ex- 
ists a controller K-o G 1Z P such that K-q stabilizes V if and only if (C 2 , A, B 2 ) is stabilizable 
and detectable. In this case, one such controller is 
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where K and L are chosen such that A + B 2 K and A + LC2 are Hurwitz. 
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We are now ready to state the analogous result in the triangular case. 

Theorem 5. Suppose V € 1Z P and V22 € lower(7\L p , k, to). Let (A, B, C, D) be a minimal 
realization of V that satisfies (5) and (6). There exists JCq e lower (TZ P , m, k) such that 
/Co stabilizes V if and only if both 

(i) (C11, A\\, B\i) is stabilizable and detectable, and 

(ii) (C22, ^22, B22) is stabilizable and detectable. 
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In this case, one such controller is 



JC a — 



A + B 2 K d + L d C 2 + L d D 22 K d 



K d 



(7) 



whe 



K d ± 



and L d = 



and Ki and Li are chosen such that An + BnKi and An + LiCu are Hurwitz for i — 1,2. 

Proof. ( ) Suppose that (i) and (ii) hold. Note that A + B 2 K d and A + L d C 2 are 
Hurwitz by construction, thus (C 2 ,A, B 2 ) is stabilizable and detectable. By Proposition 4, 
the system V can be stabilized, and a stabilizing controller is given by (7). Due to the 
block-diagonal structure of K d and L d , it is straightforward to verify that /Co € lower(7^. p ). 
( =>■ ) Suppose a controller Kq € lower (1Z P , m, k) stabilizes V. By Proposition 4, the 
realization (C 2 ,A,B 2 ) is stabilizable and detectable. By Theorem 1, we may assume 
that ICq has a minimal realization with Ax , Bk , Ck, Dk € lower (R). Proposition 3 now 
implies that I — D 22 Dk is invertible. Since D 22 and Dk are block-lower-triangular, write 
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22 
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and Dk — 
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Hence we can conclude that I — D\\Dk\\ and I — D 22 Dkti are both invertible. Secondly, 
Proposition 3 also implies that A is Hurwitz. Transform A using a matrix T that permutes 
states 2 and 3, and the resulting matrix is block-lower-triangular. Now T _1 AT is Hurwitz 
implies that the two 2x2 blocks on the diagonal are Hurwitz. But these diagonal blocks 
are precisely the A matrices corresponding to the 11 and 22 subsystems. It follows from 
Proposition 3 that (Cu, An, Bu) is stabilizable and detectable for i = 1, 2. ■ 

Note that in Theorem 5, the conditions for stabilization are stronger in the decentralized 
case than those of Proposition 4 for the centralized case. Of course, this is because we have 
imposed the additional sparsity constraint on the controller. It is also worth observing 
that one can use the same argument to show that, if there exists a block lower-triangular 
stabilizing controller, then there also exists a block diagonal stabilizing controller. We 
also characterize the stability of just the 22 block, which we state as a corollary. 

Corollary 6. Suppose V 22 € lowcr(7£ p , k, m), and let (A, B 2 ,C 2 , D 22 ) be a minimal 
realization of V 22 that satisfies (6). There exists JCo € lower (1Z P , m, k) such that /Co 
stabilizes V 22 if and only if both 

(i) (Cii) Ai\, Bu) is stabilizable and detectable, and 

(ii) {C 22 , A 22 , B 22 ) is stabilizable and detectable. 

Proof. ( 4= ) Augment 7^22 by setting V\\ = 0, V12 
realization for V is given by 



0, and V 2 \ =0. Then a minimal 
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Suppose that (i) and (ii) hold. By Theorem 5, there exists /Co G lower(7^ p ) which 
stabilizes V , and by Proposition 2, JC also stabilizes V 22 . 

( ==> ) Suppose a controller ICq € lower(7?. p , m, k) stabilizes V 22 . Since (A, B 2 , C 2 , D 22 ) is 
minimal, we have that (C 2 ,A,B 2 ) is stabilizable and detectable. The result then follows 
as in the proof of Theorem 5. ■ 
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In the centralized case, every V22 <= Hp can be stabilized. Corollary 6 shows that 
structured stabilization is nontrivial. Indeed, there exist block-lowcr-triangular transfer 
matrices that cannot be stabilized by a block-lower-triangular controller. For example, 
consider 
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The above realization is minimal, but the grouping of states into blocks is not unique. We 
can choose to group the unstable mode cither in the An block or in the A 2 2 block, which 
corresponds to n = (2, 1) or n = (1, 2), respectively. If we choose the former, it will lead 
to an undetectable (Cn,in), and if we choose the latter, it will lead to an unstabilizable 
(A22, £22). S° by Corollary 6, this plant cannot be stabilized by a block-lower-triangular 
controller. 

One may be tempted to think that a plant containing unstable poles in its off-diagonal 
block can never be stabilized using a block-lower-triangular controller, but this is false in 
general. An example is given by the plant and controller pair below. 
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We now turn our attention to the parameterization of all stabilizing controllers. 

Theorem 7. Suppose the conditions of Theorem 5 hold, and (Cu,Au,Bn) and 
(C22, A22, B22) are stabilizable and detectable. Define Kd and Ld as in Theorem 5. The 
set of all JC G lowcr(7^ p , m, k) that stabilize V is 

{Fe(Jd, Q) I Q € lower (7^"H oo, m, k) and I + D 2 2Q(oo) is nonsingular} 

where A d = A + B 2 K d + L d C 2 + L d D 2 2K d and 





r A d 


-L d 


B 2 + L d D 22 


Jd = 


K d 





I 




_ -(C 2 + D 22 K d ) 


I 


-D 22 



(8) 



Proof. If we relax the constraint that Q be lower triangular, then this is the standard 
parameterization of all centralized stabilizing controllers [30, Theorem 12.8]. It suffices 
to show that the map from Q to JC and its inverse are structure-preserving. Since each 
block of the state-space realization of J d is in lower (R), we have {Jd)ij € lower(7^ p ). 
Thus Fi( J dl ■) preserves lower triangularity on its domain. This also holds for the inverse 
map F U {J^ X , ■), since 
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-L d 


J d - X = 


c 2 


D22 
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As in the centralized case, this parameterization of stabilizing controllers allows us to 
transform the controller synthesis problem into a stable model-matching problem. For 
the two-player problem, the structural constraint we have on the controller will amount to 
the same structural constraint on the Youla parameter. We have the following algebraic 
result. 
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Theorem 8. Suppose the conditions of Theorem 7 hold, Q € lower(7\L'H 00 , to, k) and the 
matrix I + D 22 Q{co) is nonsingular. Then if JC = F t {J dl Q) we have 

where J d is given by (8), and T is 









" A + B 2 K d 




B 1 


£2 


'Tii 


T12 







A + L d C 2 







T21 


T 22 _ 




Ci + D 12 K d 


-D 12 K d 


£>n 













c 2 


£>21 






Proof. This follows from simplifying Fi(P, Fi(J d , Q)). ■ 
Combining the previous two results gives the following important corollary. 
Corollary 9. Suppose the conditions of Theorem 7 hold. Suppose Q opt is a minimizer for 

minimize \\Tu + T12QT21 1| 2 

subject to Q€ lower (7?.% oo, to, k) (10) 
I + D 22 Q{co) is invertible 

Then choosing /C opt = Ft{J d , Q op t) 5«wes a minimizer for the the two-player output feed- 
back problem (4). _ffere J d is given by (8), andT is defined in (9). Furthermore, <2 opt is 
unique if and only if JC op t is unique. 

Remark 10. In the case where V is already stable, a stabilizing controller is found by 
choosing K d = and L d = 0. The parameterization of Theorem 7 simplifies to 

IC = F e (J d ,Q) = Q(I + V 22 Q)- 1 (11) 

Finally, the result of Corollary 9 amounts to solving the model-matching problem where T 
is replaced by V , and once we have found Q op t, we can recover /C op t via (11). 

Using a parameterization akin to that of Youla, Corollary 9 transforms the two-player 
output feedback problem (4) into a two-player stable model-matching problem (10). The 
new formulation should be easier to solve than the output feedback version because it is 
convex and the %j are stable. However, its solution is still not straightforward, because 
the problem remains infinite-dimensional and there is a structural constraint on Q. In 
the next section, we will show how to solve this problem, and find the solution to (4) in 
the process. 



IV Main result 

In this section, we present our main result: a solution to the two-player output feedback 
problem. First, we state our assumptions and list the equations that must be solved. Our 
first assumption is that the plant V satisfies 



P11 Vx2 

V21 v 22 



A 


Bx 


B 2 I 




c\ 





D 12 


is a minimal realization 


c 2 


D21 








(12) 



in which the matrices A, B 2 , and C 2 have the form 



A n 
A21 





-422 



Bo 



B n 
B 2 \ 





B 22 



C 2 ^ 



C n 

C21 C22 



(13) 
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where A n G K™ 1 *™ 1 , B n G M n i xm i and C n G R felXni . If P 22 G lower(ft p , fc, m), then 
there is no loss in generality in assuming this structure for A, B 2 ,C 2 because of Theo- 
rem 1. We further assume that An and A22 have non-empty dimensions. This avoids 
trivial special cases and allows us to streamline the results. As for the assumptions 
that Dn = and D22 — 0, these assumptions can be made without loss of generality as 
in the centralized case [30]. To ease notation, we define the following cost and covariance 
matrices 

,-.t .-. r , 

[Ci i?i 2 ] T [Ci Di 



Q 


s~ 






CjD 12 ~ 


s T 


R 




DJ2C1 


Dj 2 D 12 


w 


u 1 ' 






BiDj, 


u 


V 




D 21 Bj 


D 21 D T 21 



'12 













D21 




£>21. 



(14) 



The main assumptions for the two-player problem are as follows. 
Al) Dj 2 D 12 > 

A2) (A\i,B\\) and (A 22 ,B 22 ) are stabilizable 



A3) 



A - juil B 2 
Ci ^12 



has full column rank for all lu G 



A4) D 2l D T 2l > 

A5) (Cii,Au) and (C 22 ,A 22 ) are detectable 



A6) 



A - jul Bx 
C2 D21 



has full row rank for all u> G 



We will also require the solutions to four AREs 

(X, K) = ARE(A, B 2 , C\,Di 2 ) (Y, L T ) = ARE(A T , C 2 , Bj , D 21 ) 



(15) 



(X,J) = AKE(A 22 ,B 22 ,C 1 E 2 ,D 12 E 2 ) (Y, M T ) = ARE(A T n , Cj l7 EjBj, EjD J 21 ) 
Finally, we must solve the following simultaneous linear equations for $ and >3/ 

(A 22 + B 22 J) T $ + $(i u + MC U ) -(X- A 22 )(*C 1 T 1 + U^Vf^Cu 

+ (XA21 + J T Sj 2 + Q21 - XnMCu) = (16) 



(A 22 + B 22 J)y + *(An + MC n ) T - B 22 R 22 \Bj 2 ^ + Sj 2 )(Y - Y u ) 

+ (A 21 Y + Uj 2 M T + W21 - B 22 JY 21 ) = (17) 

and define the associated gains K and L 



K = 



0' 

-R 22 x (B 2 T 2 $ + Sj 2 ) J 



L = 



M 



(18) 



The matrices <E> and VE" have physical interpretations, as do the gains K and L. These will 
be explained in Section V. The main result of this paper is Theorem 11, given below. 

Theorem 11. Suppose V G 1Z P satisfies (12), Assumptions A1-A6, and the structural 
condition (13). The two-player output feedback problem 

minimize \\V U + V 12 JC(I - V 22 K)- X V 2X || 2 
subject to JC G lower (7£ p , m, k) 
JC stabilizes V 
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as stated in (4) has a unique solution, given by 





' A + B 2 K + LC 2 





-L ' 




^opt — 


B 2 K - B 2 K 


A + LC 2 + B 2 K 


-L 


(19) 




K-K 


k 








where K, L, k and L are defined in (15) -(18). 

Note that the sparsity structure of the gains L and k gives JC op t the desired block- 
lower-triangular structure. In the next sections, we will discuss several aspects of this 
solution. First, a brief discussion on duality and symmetry. We then show a structural 
result, that the states of the optimal controller have a natural interpretation as minimum- 
mean-square-error estimates. This will lead to a useful interpretation of the matrices $ 
and that were computed in (16) and (17). We then compute the H^norm. of the 
optimally controlled system, and characterize the portion of the cost attributed to the 
decentralization constraint. Finally, we show how our main result specializes to many 
previously solved cases appearing in the literature. 

We provide a complete proof to Theorem 11 below. First, motivated by Corollary 9, we 
will consider model-matching problems with triangular constraints. We show that under 
our assumptions, an optimal rational controller exists and is unique. We construct the 
optimality conditions for the two-player problem and show that (19) satisfies them. 

We consider model-matching problems that have stable parameters. Stability is im- 
portant because it allows us to work in the Hilbert space 'H 2 and derive useful optimality 
conditions. For the rest of this section, we assume the %j € 1Z'H 00 satisfy 

Bl) Tn(oo) = 

B2) T\ 2 {joj) has full column rank for all to e R U {oo} 

B3) T 2 i{]uj) has full row rank for all cu e M U {oo} 

Model-matching problems with stable parameters have associated optimality conditions 
that follow from the Hilbert space projection theorem. The optimality condition for the 
centralized version of the problem is given in the following lemma. 

Lemma 12. Suppose T € TZHoo satisfies Assumptions B1-B3. Then 

minimize \\T\i + T\ 2 QT 2 \ |L , 

" 112 (20) 

subject to Q € VJH 2 

has a unique solution. Furthermore, Q is the minimizer of (20) if and only if 

T* 2 (Tn + T12 Qhi) T 2 \ e H 2 (21) 

Proof. Consider (20) with the relaxed constraint that Q € W. 2 instead of Q G TZV. 2 . 
The map Q T\ 2 QT 2 \ is a bounded linear operator from H 2 to H 2 , so the Hilbert space 
projection theorem (see, for example [13]) implies that (21) is necessary and sufficient for 
optimality. If Q £ 1ZH 2 is a solution to (20), then the density of rationals imply that 
it must also be a solution to the relaxed problem and thus it satisfies (21). Conversely, 
solutions to (21) that happen to be rational must also be solutions to (20). Existence of 
an optimal rational controller follows from Proposition 24, while uniqueness follows from 
Assumptions B1-B3. ■ 

Note that in Corollary 9, the constraint was Q e TZHoo rather than Q e TZV. 2 . 
There is no loss of generality in assuming Q G TZV. 2 because of Assumptions B1-B3. 
Also <2(oo) = is necessary to ensure we have a finite 'H 2 - n orm. In the following lemma, 
we develop an optimality condition similar to (21) for the two-player model-matching 
problem. 
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Lemma 13. Suppose T € IZHoo satisfies Assumptions B1-B3. Then the two-player 
model- matching problem 

minimize \\Tii + T12QT21 |L , 

" 112 (22) 

subject to Q € lower(7^'H2) 

has a unique solution. Furthermore, Q is the minimizer of (22) if and only if 



V 2 {Tn+T 12 QT 21 )T 2 \ e 



<-2 *~2 
(■2 %2 



(23) 



Proof. Since the "H 2 -norm is invariant under rearrangement of matrix elements, we may 
vectorize [6] the contents of the norm in (22) to obtain 

minimize ||vec(7n) + {T 2 \ ® T12) vec(Q) || 2 
subject to Q € lower (7£%2) 

Due to the sparsity pattern of Q, some entries of vec(Q) will be zero. Let E be the identity 
matrix with columns removed corresponding to these zero-entries. Then Q € \owct(TZ'H 2 ) 
if and only if vec(Q) = Eq for some q e TZV.2- Then (24) is equivalent to 

minimize ||vec(7u) + (T 2 \ ® Ti2)Eq\\ 2 

(25 ) 

subject to g€ IZH2 

This is a centralized model-matching problem of the form (20). Using standard properties 
of the Kroncckcr product, one may verify that Assumptions B1-B3 are inherited by 

vec(Tu) {T 2 \®Ti 2 )E 
1 

The optimality condition (23), along with existence and uniqueness of the solution, follows 
from Lemma 12. ■ 

The vectorization approach of Lemma 13 not only proves the existence of an optimal 
controller, it provides a reduction to a centralized problem with a well-known solution; 
see Proposition 24. Unfortunately constructing the solution in this manner is not feasible 
in practice because it requires finding a state-space realization of the Kroncckcr product 
system. This leads to a dramatic increase in state dimension, and requires solving a 
large Riccati equation. Furthermore, we lose any physical interpretation of the states, as 
mentioned in Section I. 



Proof of Theorem 11. To verify that JC opt is optimal, use the parameterization of 
Theorem 7 to find the parameter Q s that generates /C opt . The computation yields 



Q s — Fu{J d 1 ,^C op t) 



A + 


B 2 K 





L 







A + LC 2 


L d -L 


K d 


- K 


k 






We notice that Q s € lower(7?.H2), so by Theorem 7, JC opt is an admissible stabilizing 
controller. Now apply Lemma 13 to the T found by the stabilizing transformation (9). 
After some algebraic manipulation and simplifications, we can verify that 



T 1 * 2 (Tu + T 12 Q s T 2 i)T 2 * 1 g 



W-2 



£ 2 
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so Q s is the solution to the induced two-player model-matching problem. (A similar 
algebraic approach is detailed in Section VI.) Furthermore, it is straightforward to check 
that Assumptions B1-B3 hold in the induced problem as well. It follows by Corollary 9 
that /C pt is the unique optimal controller for the two-player output feedback problem. ■ 

The optimal closed-loop map is 

Vu + V 12 IC opt (I - TWCoptr 1 ^! = Tn + TizQsTai 

This result significantly differs from the centralized case in the following way. In the 
centralized case, choosing the Youla parameter using K and L rather than Kd and Ld 
yields a nominal stabilizing controller that also happens to be "H 2 -optimal. So we find 
that the optimal centralized controller K, centr corresponds to <2 you = 0- This is not the 
case for the two-player problem because of the additional structural constraint. Indeed, 
the nominal stabilizing controller from Theorem 5 has the same state dimension as the 
plant, whereas the optimal controller has twice the state dimension of the plant. 



V Structure of the optimal controller 
V-A Symmetry and duality 

The solution to the two-player output feedback problem presents some nice symmetry 
properties that are perhaps unexpected given the network topology. Player 2 obtains 
more information than Player 1, so one might expect Player 2's optimal control policy 
to be more complicated than that of Player 1. Yet, this is not the case. While Player 2 
observes all the measurements, he only controls u 2 , so his influence is limited to his own 
system. In contrast, Player 1 only observes his own system, but he controls m, which in 
turn influences both systems. This duality is reflected in (15)— (18). For convenience, we 
define the Hurwitz matrices 



A K ^A + B 2 K A L ^A + LC 2 

Aj 4 A 22 + B 22 J A M = A n + MC n 

A = A + B 2 K + LC 2 



(26) 



Note that Ak,Al,Aj,Am arc all Hurwitz by construction, and A is Hurwitz as well, 
because it is block- lower-triangular and its block-diagonal entries are Am and Aj. 

If we transpose all the system variables and swap Player 1 and Player 2, then every 
quantity related to control of the original system becomes a corresponding quantity related 
to estimation of the transformed system. More formally, if we define the operator 



A* = 



A T A T 

^22 ^12 
A T A T 



then the transformation (A, B 2 ,C 2 ) i-> {A^,C\,B\) leads to 

{X,K)^(Y t ,L t ), (X,K)^(Yt,L*), and A^AK 
Note as well that the realization (19) has a dual representation given by 





A + B 2 K + LC 2 





L 


^opt — 


LC 2 — LC 2 


A + LC 2 + B 2 K 


L-L 




- K 


-k 






(27) 



15 



V-B Estimation structure 

Recall the plant equations, written as differential equations 

x = Ax + B\w + B 2 u 
y = C 2 x + D 21 w 



(28) 



The optimal controller (19) is given in Theorem 11, and we label its states as £ and £, 
and place each equation into canonical observer form. 



C = A( + B 2 u - L(y - C 2 () 
u = K( 

i = Ai + B 2 u- L(y - C 2 



(29) 
(30) 
(31) 
(32) 



This choice of coordinates leads to a natural interpretation, that ( and £ are the states 
of steady-state Kalman filters that estimate the plant's state x. This claim will be made 
precise in the development that follows. 

Lemma 14. We make use of the definitions of (12) -(18) and (26). There exists a unique 
matrix Y satisfying the Lyapunov equation 



A(Y -Y) + (Y- Y)A T + {L- L)V(L - L) J 
Further, Y >Y, Y n = Y, Y 21 = and 

L = -{yd + U T )E 1 V n 1 Ej 







(33) 



(34) 



Proof. Since A is stable and (L — L)V(L — L) T > 0, it follows from standard properties 
of Lyapunov equations that (33) has a unique solution Y — Y > 0. Right-multiplying 
by Ei gives 



A(YE X 



YE!) + {YE 1 - YE^A'm 



(L- L)V(E 1 M T -L7e x ) 







Upon comparison with (15), (17), and (18), one can verify that this equation is satisfied 
by Y\\ — Y and Y 2 \ = Similarly comparing with (18) verifies (34). ■ 

Note that (34) provides a way of computing L that is equivalent to our original defini- 
tions in (18). Our next observation is that under the right coordinates, the controllabil- 
ity Gramian of the closed-loop map is block-diagonal. 

Theorem 15. Suppose we have the plant described by (28) and the optimal controller 
given by (29) -(32). The closed-loop map for one particular choice of coordinates is 



w 



which we write compactly as q = A c q + B c w. Let O be the controllability Gramian, i.e. 
the solution > to the Lyapunov equation A C Q + QAj + B c Bj = 0. Then 



c 




Ax —LC 2 —LC 2 




C ' 




-LD 21 


i-c 




A (L — L)C 2 




£-C 


+ 


(L - L)D 21 






_ A L 








B x + LD 21 



e 



z o 

Y — Y 




where Z satisfies the Lyapunov equation 

A K Z + ZA\ + LVL T = 



16 



Proof. Uniqueness and positivity of follows because A c is Hurwitz. The solution can 
be verified by direct substitution and by making use of the identities in Lemma 14. ■ 

The observation in Theorem 15 lends itself to a statistical interpretation. If w is IID 
white Gaussian noise, and we consider the steady-state distribution of the random vec- 
tor q, its three components are mutually independent. Since £ is the sum of the first 
two states, we have that £ and x — £ are independent, and similarly £ and x — Q are also 
independent. This fact supports our claim that £ and £ are estimators for x, and we will 
provide a proof in the development that follows. 

As expected from our discussion on duality, the estimation structure discussed 
in Lemma 14 and Theorem 15 above has an analogous control structure. We state the 
analogue to Lemma 14 without proof. 

Lemma 16. We make use of the definitions of (12) -(18) and (26). There exists a unique 
matrix X satisfying the Lyapunov equation 

A T {X -X) + (X- X)A + (K- K) T R(K -K) = (35) 

Further, X > X, X 22 = X, X 21 = and 

K = -E 2 R 22 1 E](B^X + S T ) (36) 

The dual of Theorem 15 can be obtained by expressing the closed-loop map in the 
coordinates (x, x — £, x — £) instead. In these coordinates, one can show that the observ- 
ability Gramian is block-diagonal. 

Now, we prove some structural properties of the optimal controller. Most importantly, 
we show that the states £ and £ of the optimal controller can be interpreted as Kalman 
filters. We begin by formally defining a notation that was first used in the introduction. 

Definition 17. Suppose Q E Ti { J >1+P2)x{qi+q2) is given by 

g _ Gil Gl2 

Qi\ Gii_ 

and G 2 i(juo) has full row rank for all u) G 1U {oo}. Define the estimator of Q to be 
g cst e K p p lX{p2+qi) partitioned according to g cst = [0f st ^ st ] where 

argmin \\g n - TG 2 i |L 

Gi2 - GTGii 




Note that under the assumptions of Definition 17, the estimator £ ost is unique. We will 
show existence of G est for particular Q below. We now define the following notation. 

Definition 18. Suppose Q and G est are as in Definition 17. If 



X 




w 




= 9 








u 



then we use the notation x\ y u to mean 
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This notation is motivated by the property that 

x-x ]y , u = {g 11 -gi at g 21 ) w 

and hence we are choosing C?J st to minimize the mean square estimation error. In the 
following lemma we compute the estimator from Definitions 17 and 18 for the centralized 
case and show that this is exactly the classical steady-state Kalman filter. Specifically 
this result shows that x\ y u is the usual steady-state Kalman filter for estimating the 
state x using the measurements y and inputs u. 

Lemma 19. Let Q be 





- A 




B 2 


Q = 


I 










. c 2 


£>21 






and suppose Assumptions AJ±-A6 hold. Then 

Qest 



A L 


-L B 2 


I 






where L is given by (15). 

Proof. Let Qf st and L be as above. For the optimization problem of Definition 17, 
change variables according to T = Qi st +T. Then, in these coordinates, the optimization 
problem is 



minimize 



A L 


B 1 + LD 21 


I 






TQ 2 



subject to T e 1ZH 2 



together with the additional constraint that the quantity in the norm is stable. For 
this proof, it suffices to show that T = is optimal. We begin by parameterizing all 
T € VM. 2 such that the quantity in the norm is stable. By [30, Thm. 13.35], the following 
factorization is coprime 



A L 


L ' 


-l 


' A L 


B x + LD 2l ' 


c 2 


I 




c 2 


D 21 



Qi\ = 



and AfAf* = L^i-Dji- Hence the set of T E TZH 2 such that 

Tg 21 e nn 2 



= M~ l M 



(37) 



is parameterized by T — QM where Q € TZV. 2 . To see why, note that if T e VSH. 2 
and (37) holds then 

T [I M- X N] e KH 2 =^> TM- 1 [M TV] e TUi 2 =^> TMr 1 e KH 2 
So T = QM for some Q e KH 2 . Conversely, if T = QM, then 
-F^i = {QM)(M- 1 U) = QAfe HU 2 
So finally, our optimization problem takes the following unconstrained form 



minimize 



A L 


By + LD 21 


- Q 


A L 


B 1 + LD 21 


I 





c 2 


D 21 



subject to Qe TZV. 2 
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We conclude that Q = is optimal, and hence T = 0, because 



A L 


B 1 + LD 21 




A L 


B x + LD 21 


I 







c 2 


D 2 i 



(38) 



Comparing the result of Lemma 19 with the £-state of the optimal two-player con- 
troller 31, we notice the following consequence. 

Corollary 20. Suppose the conditions of Theorem 11 are satisfied, and label the states 
of the controller as in (29) -(32). Then 

where the estimator is defined by the map Q : (w,u) h-> (x, y) induced by the plant (28). 

The above result means that £, one of the states of the optimal controller, is the usual 
Kalman filter estimate of x given y and u. The next result is more difficult, and gives the 
analogous result for the other states. The next theorem is our main structural result. We 
show that £ may also be interpreted as an optimal estimator in the sense of Definition 17 
and 18. 

Theorem 21. Suppose the conditions of Theorem 11 are satisfied, and label the states of 
the controller as in (29) -(32). Define also 



(39) 



so that u = + U£. Then 



X 




"c" 






c 


u 


1 !/!.«< 


it 







Here the estimator is defined by the map Q : (w,u^) 1— !► (x,£, u, yi) induced by (39), the 
plant (28), and the controller (29) -(32). 

Proof. The proof parallels that of Lemma 19, so we omit most of the details. Straight- 
forward algebra gives 



A 


b 2 k 


Si 


B 2 ' 


-LC 2 A - 


- LC 2 + B 2 K 


-LD 21 


B 2 


I 














I 











k 





I 


eJc 2 





EjD 21 






After substituting T = £J st + J 7 , computing the coprime factorization, and changing to 
more convenient coordinates, the resulting unconstrained optimization problem is 



minimize 



A 


(L - L)C 2 


(L 


- L)D 21 - 







Ah 


B 2 


+ LD 21 




1 


I 







- QV 


1 












K 













subject to Qe TZV. 2 
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where A is defined in (26), and 





A (L - L)C 2 


{L-L)D 2U 


V = 


A L 


B 2 + LD 21 




EjC 2 EjC 2 


EjD 21 



Optimality of Q = is established by verifying the analogous orthogonality relationship 
to (38). This is easily done using the Gramian identity from Theorem 15. Then we have 





" A 


—LEi 


B 2 


QCSt 


I 










I 










. K 









Comparing with (29), we can see that 

c 
c 

u 



and the result follows. 



The result of Theorem 21 that £ = x\ yiiU( , has a clear interpretation that £ is a Kalman 
filter of x. However, this assumes that Player 2 is implementing the optimal policy for u 2 . 
This is important because the optimal estimator gain depends explicitly on this choice of 
policy. In contrast, in the centralized case, the optimal estimation gain does not depend 
on the choice of control policy. 

Note that using u\ or u instead of in Theorem 21 yields an incorrect interpretation 
of (. In these coordinates, 



A + B 2 E 2 EjK + LC 2 


— LE\ B2E1 


I 








yi 







c = 



A + B 2 K + LC 2 


— LEi B2 


I 








yi 







In general, neither of these maps is stable, so for these choices of signals, £ cannot be an 
estimator of x in the sense of Definitions 17 and 18. 

We can now state the controller in a particularly simple form. The controller is 



u = Kx 



li/i.«< 




H J 



{ x \y,u 



(40) 



where H = K 2 i . These equations make clear that Player 1 is using the same control action 
that it would use if the controller was centralized with both players only measuring y\ . 
We can also see that the control action of Player 2 has an additional correction term, given 
by the gain matrix \H J] multiplied by the difference between the players' estimates of 
the states. Note that is a function only of y\ , as can be seen from the state-space 



C = A K Q-LE 1 {y 1 
u c = (K- K)( 



[Cn 0]C) 



The orthogonality relationships of the form (38) used in Lemma 19 and Theorem 21 to 
solve the V. 2 optimization problems may also be interpreted in terms of signals in the 
optimally controlled closed-loop. 
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Corollary 22. Suppose u,w,x and y satisfy (28) -(32). Then the maps E 2 : w h-> x — £ 
and R 2 : w h-» y — C 2 £, w/iic/i groe i/ie error and residual for Player 2, are 



E 2 



A L 


Si + £A>i 


I 






i?2 





Bi + L£>2i 


c 2 


£>21 



ITie maps i?i : u> ^ x — ( and R\ : w i-» yi — CuCi, which give the error and residual 
for Player 1, are 





' A 


{L-L)C 2 


(L 


- L)D 21 


E 1 = 





A L 


Bi 


+ LD 21 




J 


I 






A M 


MEj - EjL 


. Cl1 


EJ 



^1 — ^ ^2 



Further, the orthogonality conditions E 2 R 2 € "H^ and -Ea-RJ € "H^ are satisfied. 

As with Theorem 15, Corollary 22 lends itself to a statistical interpretation. If w is 
IID white Gaussian noise, and we consider the steady-state distributions of the error and 
residual for the second player, x — £ and y — C 2 £ respectively, then they are independent. 
Similarly, the error and residual for the first player, x — £ and y\ — Cn£i, are also 
independent. 



V-C Optimal cost 

We now compute the cost associated with the optimal control policy for the two-player 
output feedback problem. From centralized H 2 theory [30], there are many equivalent 
expressions for the optimal centralized cost. In particular, 



HW^centr)!!! 



= trace(XW) + tmce(YK T RK) 
= tracc(yQ) + trace(XH/L T ) 

where X, Y, K, L are defined in (15). Of course, the cost of the optimal two-player con- 
troller will be greater, so we may write 



A K 




Ci + D 12 K 






A L 


B 1 + LD 21 


D 12 K 






\Ft{V,Ko 



pt;ii2 



\F t {V,Kc 



where A > is the additional cost incurred by decentralization. We now give some 
closed-form expressions for A that are similar to the centralized formulae above. 

Theorem 23. The additional cost incurred by the optimal controller (19) for the two- 
player problem (4) as compared to the cost of the optimal centralized controller is 



A 


(L - L)D 21 ' 


D 12 (K-K) 






A = 



= trace(y - Y)(K - K) T R(K - K) 
= trace(A - X)(L - L)V(L - L) T 

where K, K, L, L are defined in (15)-(18), X and Y are defined in (33) and (35), and 
A is defined in (26). 
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Proof. The key is to view /C opt as a sub-optimal centralized controller. Centralized H 2 
theory [30] then implies that 

A = \\D 12 Q you D 21 \\ 2 2 (41) 
where <2 you is the centralized Youla parameter. Specifically, <2 you = F U {J~ X , /C op t) and 





r A 


B 2 


-L ' 


J- l = 


c 2 





I 






I 






Since this is the centralized parameterization, it contains the optimal centralized gains K 
and L instead of Kd and Ld- After some simplification, we obtain 



Q 



you 



A 


L-L~ 


k-K 






(42) 



substituting (42) into (41) yields the first formula. The second formula follows from eval- 
uating (41) in a different way. Note that \\D S + C s (sl - A S )^ 1 B S \\ 2 = trace(C s W c Cj), 
where W c is the associated controllability Gramian, given by A S W C + W C A] + B s Bj = 0. 
In the case of (41), the controllability Gramian equation is precisely (33), and there- 
fore W c = Y — Y and the second formula follows. The third formula follows using the 
observability Gramian W Q = X — X given by (35). ■ 

Theorem 23 expresses the total cost as a sum of two parts; the cost of the optimal 
centralized controller, and the additional cost incurred by requiring the controller to be 
block-lower-triangular. This allows us to precisely quantify the cost of decentralization. 



V-D Some special cases 

The optimal controller (19) is given by (40), which we repeat here for convenience. 



u = Kx 



li/i.«< 



+ 




H J 



(%\y,u 



Recall that K and J are found by solving standard AREs (15). The coupling between 
estimation and control appears in the term H = -R^{B^ + Sj 2 ), which is found by 
solving the coupled linear equations (16)— (17) . 

Several special cases of the two-player output feedback problem have previously been 
solved. In each of them, the first component of {x\ y u — x\ yijU( ) is zero. In other words, 
both players maintain identical estimates of X\ given their respective information. Con- 
sequently, u no longer depends on H and there is no need to compute $ and We now 
examine these special cases in more detail. 

Centralized. The problem becomes centralized when both players have access to the 
same information. In this case, both players maintain identical estimates of the entire 
state x. Thus, x\ y u = x\ yitU( — x and we recover the well-known centralized solu- 
tion u = Kx. 

State- feedback. The state-feedback problem for two players is the case where our mea- 
surements are noise-free, so that y\ = x\ and y 2 — x 2 . Therefore, both players know x\ 
exactly. This case is solved in [21,22] and the solution takes the following form, which 
agrees with our general formula. 



u = K 



X\ 




~0~ 




+ 


J 



{x 2 - x 2 \i) 
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Here, x 2 |i is an estimate of x 2 given the information available to Player 1, as stated in [22]. 

Partial output feedback. In the partial-output feedback case, yi = x\ as in the state- 
feedback case, but y 2 is a noisy linear measurement of both states. In this case, both 
players know x\ exactly. This case is solved in [23] and the solution takes the following 
form (using notation from [23]), which agrees with our general formula. 



u = K 



X\ 



+ 



(X 2 \ 2 - X 2 \i) 



Dynamically decoupled. In the dynamically decoupled case, all measurements are noisy, 
and the dynamics of both systems are decoupled. This amounts to the case where A 2 i = 0, 
B 2 i = 0, C 2 \ = 0, and W, V, U are block-diagonal. Due to the decoupled dynamics, 
the estimate of x\ based on yi does not improve when additionally using y 2 . This case 
is solved in [7] and the solution takes the following form, which agrees with our general 
formula. 



K 



Xl\l 



+ 



(X 2 \2 - X 2 \l) 



Note that estimation and control are decoupled in all the special cases examined above. 
This fact allows the optimal controller to be computed by merely solving some subset of 
the AREs (15). In the general case however, estimation and control are coupled via $ 
and * in (16) (17). 



VI Construction of the optimal controller 



The proof of Theorem 11 verifies that the given formula for JC op t is actually optimal. 
In this section, we give a state-space method for constructing it, via the solution to the 
two- player model-matching problem (22). First, we give state-space versions of Assump- 
tions B1-B3. For matrices A, B, C, D, define the following conditions. 



A - jtul B 2 
Ci D 12 



CI) Dj 2 D 12 > 

C2) 

C3) D 21 D\ X > 
Ci) 



A - jujl B 1 
C 2 D 2 i 



has full column rank for all co e 



has full row rank for all uj e 



Suppose the transfer matrix T € TZHoo has realization 



7ii 7i2 
T21 



A 


Bi 


B 2 - 







D 12 


c 2 


^21 






where A is Hurwitz. 



(43) 



Then T satisfies Assumptions B1-B3 if and only if A, B, C, D satisfy Assumptions C1-C4, 
so these assumptions depend only on the transfer function, not the particular realization. 
We will borrow a great deal of notation from Section IV. We will define Q 7 R,S, W, V, U as 
in (14), and X, Y, K, L, X, Y, J, M as in (15). Note that these quantities are now defined 
in terms of the A, B, C, D from (43). 

The centralized model-matching problem (20) has a well-known state-space solution, 
which we state as a Proposition. 
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Proposition 24. Suppose T € liMoo satisfies (43) and Assumptions C1-C4- The solu- 
tion to the centralized model-matching problem (20) is 



Q, 



opt 



" A K 


B 2 K 


" 







—L 


K 


K 






where K, L are defined in (15), and Ak, Al are defined in (26). 

We will derive the solution to the two-player model-matching problem by splitting it 
into two coupled optimization problems. We show that the problem is equivalent to 
a pair of coupled centralized model-matching problems which must simultaneously be 
solved. Once we have found this solution, we apply Corollary 9 to obtain the solution to 
the two-player output feedback problem. 



Lemma 25. Suppose T G IZHc 
that Q £ lower(7£H2) and 



satisfies (43) and Assumptions C1-C4- Further suppose 

Q 



Qn 
Q21 





Q22 



Then Q is optimal for the two-player model-matching problem (22) if and only if both of 
the following hold 



IIT11 +T12QT21W2 
HT11+T12QT21H2 



mm 

z 2 i,z 22 eiz-H2 



mm 

Z 11 ,Z 2 i£K-H2 



T11+T12 
Tu+Tl2 



Q11 





Z21 


2-22 


Zu 





-2-21 


Q22 



T21 
T21 



(44) 
(45) 



Proof. We will prove this lemma by showing that the optimality condition for (22), 
which is given by Lemma 13, is satisfied if and only if the optimality conditions for (44) 
and (45) hold. Observe that 



|Tn + T12QT21II2 - || (Tu +7i 2 £dQii£ 1 T T2i) +Ti 2 ^2 [Q21 Q22] T 2 
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(46) 



The minimization problem in (44) has no structural constraint, and so (46) and Lemma 12 
imply that (44) is equivalent to 



EIV2 (Tn + T12 QT21 ) T 2 \ e [Hjr Hi] 
Similarly (45) is equivalent to 



T 1 *2(Tii+Ti2QT2i)T 2 * 1 £ 1 e 



(47) 



(48) 



It is immediate that Q satisfies both (47) and (48) if and only if it satisfies (23). ■ 

Lemma 25 shows that solving the two-player model-matching problem (22) amounts to 
solving two centralized model-matching problems that are coupled. The first one (44) is 
an optimization in which we are holding Qn fixed and optimizing over the remaining Q 2 i 
and Q22- In the following lemma, we solve this optimization problem and give a state- 
space expression for the optimal [Q21 Q22] as a function of Qn. 

To simplify the required algebra, we will assume that in the realization for T given 
by (43), the matrices A,B 2 ,C 2 have the block- lower-triangular structure of (6). There 
is no loss of generality in this assumption, because T 22 = implies that T22 is trivially 
block-lower-triangular, so we are free to transform T as we transformed V in Section III. 
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Lemma 26. Suppose 7" € TlHoo satisfies (43), Assumptions C1-C4, and the structural 
condition (6). If 



Q 



ii 



A P 


Bp 


c P 






then the unique solution to (44) is 





' A + B 2 E 2 K 1 


B 2 E 2 K 2 


B 2 (E\Cp + E 2 Kz) 





Q22] = 





A + LC 2 





—L 








A P 


BpEj 




Ki 


K 2 


K 3 






(49) 



Here the quantities K\, K 2 , and K3 are defined by 

K x = [-R^ (Bj 2 Q x + Sj 2 ) J] K 2 - [-R 2 t (i? 2 T 2 $ + Sj 2 ) J] 
K 3 — —R 22 {Bj 2 Tx + Rj 2 Cp) 

where @x, ^x, and $ are the unique solutions to the linear equations 

A T jQ x + @xA n + XA 21 + J T Sj 2 + O21 = 
A T jT x + T X A P + (Q x B n + XB 21 + J J R 21 + S 21 )C P = 
A T j$ + $A n + XA 21 + J T Sj 2 + Q 21 + T x BpCn = 

and Q, R, S are defined in (14), and X, Y, J, L are defined in (15). 



(50) 



Proof. First, we must find a joint realization for the blocks of 71 1 +712Q721- Since Qn 
is held fixed, we group it with 71 1 as we did in the proof of Lemma 25 and obtain 



71i + T\ 2 EiQ\\EjT 2 i T\ 2 E 2 
Tix 



A 





B 2 E\Cp 





B 2 E 2 





A 







B x 








BpEjC 




A P 


BpEjD 2 i 





Ct 




D\ 2 E\C p 





D\ 2 E 2 





c 2 







D 21 





A 


B\ B 2 










Ci 


L>i2 










c 2 


D 2l 











It is straightforward to check that Assumptions B1-B3 are satisfied for this augmented 
system. Now, we may apply Proposition 24. The result is that 



[Q21 Q2 2 ] opt - 



A + B 2 K 


B 2 K 


" 





A + LC 2 


—L 


K 


K 






(51) 



where {X,K) = ARE(A, B 2 , d, D 12 ) and (Y , L T ) = ARE(A T , Cj, Bj, D 21 ). One can 
check that the stabilizing solution to the latter ARE is 



Y = 




Y 




and 



L 




L 

-BpEj 
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The former ARE is more complicated, however. Examining K = — _R _1 (i? T A > + S T ), we 
notice that due to all the zeros in B, the only part of X that affects the gain K is the 
second sub-row of the first block-row. In other words, if 



X 



X\\ X\2 X13 

X-21 X-22 X-23 

X31 X 32 X 33 



then K only depends on EjXn, EjXi 2 , and EjXi 3 . Multiplying the ARE on the left 
by [Ej 0] , we can write a single equation that determines the quantities of interest. 



A BE X C P 
OA 
BpEjC A P 



A22 [EjXu EJX12 EjXi 3 ] + [eJXh EJX12 EjXi 3 ] 

— [E2X11E2B22 + S22) R22 {^22 \E2X\1 EJX12 EjXizj 

+ [E 2 V S T EjS T R 2 iCp\) + [EjQ EjQ S 2 iC P ] = (52) 
Right-multiplying by [Ej 0] or by [0 Ej 0] , we obtain an equation of the form 

A 22 X + XA22 — {XB22 + >S'22)-R22 1 (^-^22 + S < 22) T + Q22 = 

and this is simply the control ARE corresponding to the 22 subsystem x 2 = A 2 2X2+B 2 2U2- 
Therefore, we conclude that EjXnE 2 — EjXi 2 E 2 = X. The corresponding gain is 
J = — i?^ (Bj 2 X + Sj 2 ). Equation (52) simplifies to 

[A BE X C P 
Aj [-^J^ii E\X\2 -EJA13] + [i?jAn EJX12 SJA13] OA 

BpEjC A P 

+ J T [ElS 1 E]S T R 2 iC P ] + [EjQ EjQ 5 2 iC P ] - (53) 

Notice that (53) is linear in the X terms. Assign the following names to the missing 
pieces 



£ 2 T ^n = [ex x] 



EjX 12 4 [$ X ] 



EjXi 3 — T x 



Upon substituting these definitions into (53) and simplifying, we obtain (50). The asso- 
ciated control gain K is given by K = [K 1 K 2 , where the individual components 
are defined by 

Ki = [-R22 1 (Bl 2 Qx + Sj 2 ) J] K 2 ^[-R22 1 ( B 22^ + SJ 2 ) J] 
K 3 — — i? 22 1 (bJ 2 t x + Rj 2 Cp) 

Now, substitute K and L into (51). The result is a very large state-space realization, 
but it can be greatly reduced by eliminating uncontrollable and unobservable states. The 
result is (49). This reduction is not surprising, because we solved a model-matching 
problem in which the joint realization for the three blocks had a zero as the fourth block. 



We may solve (45) in a manner analogous to how we solved (44). Namely, we can 
provide a formula for the optimal Qn and Q21 as functions of Q 2 2- The result follows 
directly from Lemma 26 after we make a change of variables. 
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Lemma 27. Suppose T € TlHoo satisfies (43), Assumptions C1-C4, and the structural 
condition (6). If 



Q22 — 



Aq 


Bq 








then the unique solution to (45) is 







A + L 1 EjC 2 








u 






L2EJC2 


A K 





L2 


Q 2 i_ 




(L 3 EJ + B Q E])C 2 





Aq 


u 









-K 


E2CQ 






(54) 



M 

+ U&)V{ : 



The quantities L\, L 2 , and L 3 are defined by 

T - \ M 1 T 

L 3 = -(T Y C'J 1 +B Q V2 1 )V 1 - 1 1 

where &y, ^y, and ^ are the unique solutions to the linear equations 

A 22 e Y + e Y A T M + (A 21 Y + Uj 2 M T + W21) = 
A Q T Y + T Y A T M + B Q (V 2 iM T + U21 + C21Y + C 2 2Qy) = (55) 
A 22 ^> + *A T M + (A 21 Y + Uj 2 M J + W21 + B 2 2CqT y ) = 

and W, V, U are defined in (14), and X, Y, K, M are defined in (15). 

A key observation that greatly simplifies the extent to which the two optimization 
problems (44) and (45) are coupled is to notice that the Qu have simple state-space 
representations. 

Remark 28. // we isolate Qu from (54) by multiplying on the left by Ej , we 
have EJE2CQ = 0, and the state-space realization can be significantly reduced. A similar 
procedure can be applied to (49) to isolate 0,22- After simplification, we obtain 



(56) 





Ar L 2 Cii 


L 2 ' 




- Aj B22K2 





Qu = 


A M 


M 


0.22 — 


A L 


—LE2 




-EjK 







J K 2 






Consequently we come to a surprising conclusion, that the modes of Q 2 2 do not depend 
on Qu. The entirety of Q22S dependence on Qu is encapsulated in K 2 . Similarly Qn 
only depends on Q22 through L 2 , and its modes are independent as well. 

Lemmas 26 and 27 provide solutions to (44) and (45). It remains to enforce the con- 
straint that these solutions be consistent. 

Theorem 29. Suppose T G TlHco satisfies (43), Assumptions C1-C4, and the structural 
condition (6). The solution to the two-player model-matching problem (22) is 



Q 



opt 



A K 


-LC 2 





-L ' 





A 


-b 2 k 


L 








A L 


L 


K 


k 


-k 






(57) 



where K, L, k, L are defined in (15)-(18) and Ak,Al,A are defined in (26). 
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Proof. First, we take care of existence and uniqueness. From Lemma 13, (22) has a 
unique rational solution. By Lemma 25, there must exist a unique simultaneous rational 
solution to (44) and (45), and it must be given by the expressions in Lemmas 26 and 27. 

We enforce that (54) and (49) match by first enforcing that Qn agrees. Substitut- 
ing (56) into Lemma 26 gives 



A P 


Bp 


C P 






A K 


L2C11 


U ' 





A M 


M 


-EjK 









The resulting system of equations (50) then becomes 

A T ,Q X + Ox A u + (XA21 + J T Sj 2 + Q 21 ) = (58) 
+ (e x B n + XB 21 + J T R 21 + S 2 i) [-EjK 0] - (59) 



A T jT x + T x 



A K 




L2C11 
A M 



A T j<$> + &A U + (XA 21 + J T Sj 2 + Q21 + T x 



L 2 
M 



Cy. 



(60) 



Note that (58) is independent of the other two equations, and uniquely determines Qx 
because it is a Sylvester equation and both Aj and An are stable. It can be verified by 
direct substitution that the solution to (59) is given by 

T x = [Q x -X21 X- X 22 $ - Q x ] 

Substituting Tx into (60) and simplifying, we obtain an equation for $ 

A T j<f> + $A U + (XA21 + J J Sj 2 + Q21) + [$ - X 2 i X- X 22 ] L 2 Cn = 

Note that this equation also contains L2, which only depends on vp. We can make the 
dependence on $ and ^> more explicit by expanding L 2 and collecting terms. Doing so, 
we obtain (16). 

Next, we follow a similar procedure to enforce that Q 22 matches. Substituting equa- 
tion (56) into Lemma 27, we find 

Y-Yn 
Qy - Y 21 



and obtain an equation for ^ that contains K 2 

A 22 ^> + VA T M + (A 21 Y + Uj 2 M T + W21) + B 22 K 2 



Y-Yn 

*-l21 







Substituting the definition for K 2 , we obtain (17). Finally, we return to (49) and substi- 
tute in the expression we found for Fx and our expression for Cp. After some simplifi- 
cation, Qx cancels and upon defining L = L 2 Ej and K = E 2 K 2 , we obtain (57). Using 
a similar procedure substituting Ty into (54), we also obtain (57), which confirms that 
both (44) and (45) have been simultaneously solved. By Lemma 25, <2 op t is the solution 
to the two-player model-matching problem. 

The existence and uniqueness properties verified at the beginning of the proof imply 
that there exists unique <I> and * which simultaneously satisfy (16)-(17). ■ 
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VII Summary 



In this article, we studied the class of two-player output feedback problems with a nested 
information pattern. We began by considering the problem of realization of block lower- 
triangular systems, and showed that, surprisingly, no additional states were required when 
imposing the constraint that the state-space matrices for such systems be triangular. 

We then presented an extensive state-space treatment of stabilization, giving necessary 
and sufficient conditions for the existence of a structured stabilizing controller. Here also 
there are surprises. In the centralized case, every plant may be stabilized in the input- 
output sense. However, there are plants for which there is no decentralized stabilizing 
controller. We then address the construction of a Youla-like parameterization of all such 
controllers. Instead of an input-output formulation, we give a state-space parameteriza- 
tion of all stabilizing controllers. This gives an immediately applicable and computable 
result, which we subsequently use for controller synthesis. 

The main result of this paper is the explicit state-space formulae for the optimal H2 
controller for the two-player output feedback problem. Just as in the centralized case, one 
is interested in more than just a formula. In the centralized case, it is a celebrated and 
widely-generalized result that the controller is a composition of an optimal state-feedback 
gain with a Kalman filter estimator. Our approach generalizes both the centralized for- 
mulae and this separation structure to the two-player decentralized case. We show that 
the "H 2 -optimal structured controller has twice the state dimension of the plant, and we 
give intuitive interpretations for the states of the controller as steady-state Kalman fil- 
ters. The player with more information must duplicate the estimator of the player with 
less information. This has the simple anthropomorphic interpretation that Player 2 is 
correcting mistakes made by Player 1. The controller also has a certainty equivalence 
property, so that if the state-estimates are replaced by the true states then the control 
action is precisely that which would be taken by the optimal centralized state-feedback 
controller. 

Both the state-space dimension and separation structure of the optimal controller were 
previously unknown. While these results show that the optimal controller for this problem 
has an extremely simple state-space structure, not all such decentralized problems exhibit 
such pleasant behavior. One example is the two-player fully-decentralized state-feedback 
problem, where even though the optimal controller is known to be linear and rational it 
is shown in [10] that the number of states of the controller may grow quadratically with 
state-dimension. 

The formulae that we give for the optimal controller are simply computable, requiring 
the solution of four standard algebraic Riccati equations, two that have the same dimen- 
sion as the plant and two with a smaller dimension. In addition, one must solve two linear 
matrix equations. All of these computations are simple and have readily available exist- 
ing code. 

We analyze the cost of the optimal controller and give an expression that explicitly 
shows the cost incurred by decentralization. The excess cost is equal to the H2 norm of 
a specific auxiliary system. 

While there is as yet no complete state-space theory for decentralized control, in this 
work we provide solutions to a prototypical class of problems which exemplify many of 
the features found in more general problems. It remains a fundamental and important 
problem to fully understand the separation structure of optimal decentralized controllers 
in the general case. While we solve the two-player triangular case, we hope that the 
solution gives some insight and possible hints regarding the currently unknown structure 
of the optimal controllers for more general architectures. 
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